A Data-Driven Regularization Model for Stereo and Flow

Data-driven techniques can reliably build semantic correspondence among images. In this paper, we present a new regularization model for stereo or flow through transferring the shape information of the disparity or flow from semantically matched patches in the training database. Compared to previous regularization models based on image appearance alone, we can better resolve local ambiguity of the disparity or flow by considering the semantic information without explicit object modeling. We incorporate this data-driven regularization model into a standard Markov Random Field (MRF) model, inferred with a gradient descent algorithm and learned with a discriminative learning approach. Compared to prior state-of-the-art methods, our full model achieves comparable or better results on the KITTI stereo and flow datasets, and improves results on the Sintel Flow dataset under an online estimation setting.National Science Foundation (U.S.) (CGV 1212849)United States. Office of Naval Research. Multidisciplinary University Research Initiative (Award N00014-09-1-1051

The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Cases

This study investigates the transformative potential of Large Language Models (LLMs), such as OpenAI ChatGPT, in medical imaging. With the aid of public data, these models, which possess remarkable language understanding and generation capabilities, are augmenting the interpretive skills of radiologists, enhancing patient-physician communication, and streamlining clinical workflows. The paper introduces an analytic framework for presenting the complex interactions between LLMs and the broader ecosystem of medical imaging stakeholders, including businesses, insurance entities, governments, research institutions, and hospitals (nicknamed BIGR-H). Through detailed analyses, illustrative use cases, and discussions on the broader implications and future directions, this perspective seeks to raise discussion in strategic planning and decision-making in the era of AI-enabled healthcare

Explaining variation in brood parasitism rates between potential host species with similar habitat requirements

Host specialization evolved in many parasite-host systems. Evolution and maintenance of host specificity may be influenced by host life-history traits, active host selection by the parasite, and host anti-parasite strategies. The relative importance of these factors is poorly understood in situations that offer parasites a choice between hosts with similar habitat requirements. The common cuckoo Cuculus canorus is a generalist parasite on the species level, but individual females prefer particular host species. In reed beds of the Yellow River Delta, China, two potential hosts with similar nest characteristics, Oriental reed warblers Acrocephalus orientalis and reed parrotbills Paradoxornis heudei, breed in sympatry. We found that warblers were parasitized at much higher rates than parrotbills. Both hosts recognized and rejected non-mimetic model eggs well, indicating that they have been involved in an arms-race with cuckoos. Cuckoo eggs closely resembled warbler eggs, and such eggs were mostly accepted by warblers but rejected by parrotbills. Only warblers recognized adult cuckoos as a specific threat. Both hosts were equally good at raising cuckoo chicks. Low nest density, partial isolation by breeding time, small scale differences in nest and nest site characteristics, and high rejection rates of natural cuckoo eggs are likely cumulatively responsible for the low current parasitism rate in parrotbills. This study emphasizes the importance of integrating the study of general host life-history characteristics and specific anti-parasitism strategies of hosts across all breeding stages to understand the evolution of host specificity.submittedVersionpublishedVersio

Seeing the arrow of time

URL to conference programWe explore whether we can observe Time’s Arrow in a temporal sequence–is it possible to tell whether a video is running forwards or backwards? We investigate this somewhat philosophical question using computer vision and machine learning techniques. We explore three methods by which we might detect Time’s Arrow in video sequences, based on distinct ways in which motion in video sequences might be asymmetric in time. We demonstrate good video forwards /backwards classification results on a selection of YouTube video clips, and on natively-captured sequences (with no temporally-dependent video compression), and examine what motions the models have learned that help discriminate forwards from backwards time.European Research Council (ERC grant VisRec no. 228180)National Basic Research Program of China (973 Program) (2013CB329503)National Natural Science Foundation (China) (NSFC Grant no. 91120301)United States. Office of Naval Research (ONR MURI grant N00014-09-1-1051)National Science Foundation (U.S.) (NSF CGV-1111415

Deviation magnification: Revealing departures from ideal geometries

Structures and objects are often supposed to have idealized geometries such as straight lines or circles. Although not always visible to the naked eye, in reality, these objects deviate from their idealized models. Our goal is to reveal and visualize such subtle geometric deviations, which can contain useful, surprising information about our world. Our framework, termed Deviation Magnification, takes a still image as input, fits parametric models to objects of interest, computes the geometric deviations, and renders an output image in which the departures from ideal geometries are exaggerated. We demonstrate the correctness and usefulness of our method through quantitative evaluation on a synthetic dataset and by application to challenging natural images.Shell ResearchQatar Computing Research InstituteUnited States. Office of Naval Research (Grant N00014-09-1-1051)National Science Foundation (U.S.) (Grant CGV-1111415

RibSeg v2: A Large-scale Benchmark for Rib Labeling and Anatomical Centerline Extraction

Automatic rib labeling and anatomical centerline extraction are common prerequisites for various clinical applications. Prior studies either use in-house datasets that are inaccessible to communities, or focus on rib segmentation that neglects the clinical significance of rib labeling. To address these issues, we extend our prior dataset (RibSeg) on the binary rib segmentation task to a comprehensive benchmark, named RibSeg v2, with 660 CT scans (15,466 individual ribs in total) and annotations manually inspected by experts for rib labeling and anatomical centerline extraction. Based on the RibSeg v2, we develop a pipeline including deep learning-based methods for rib labeling, and a skeletonization-based method for centerline extraction. To improve computational efficiency, we propose a sparse point cloud representation of CT scans and compare it with standard dense voxel grids. Moreover, we design and analyze evaluation metrics to address the key challenges of each task. Our dataset, code, and model are available online to facilitate open research at https://github.com/M3DV/RibSegComment: 10 pages, 6 figures, journa